AI mobile robot for learning obstacle and method of controlling the same

ABSTRACT

An artificial intelligence (AI) mobile robot and a method of controlling the same for learning an obstacle are configured to capture an image while traveling through an image acquirer, to store a plurality of captured image data, to determine an obstacle from image data, to set a response motion corresponding to the obstacle, and to operate the set response motion depending on the obstacle, and thus, the obstacle is recognized through the captured image data, the obstacle is easily determined by repeatedly learning an image, and the obstacle is determined before the obstacle is detected or from a time point of detecting the obstacle to perform an operation of a response motion, and even if the same detection signal is input when a plurality of different obstacles is detected, the obstacle is determined through the image and different operations are performed depending on the obstacle to respond to various obstacles, and accordingly, the obstacle is effectively avoided and an operation is performed depending on a type of the obstacle.

CROSS-REFERENCE TO RELATED PATENT APPLICATIONS

This application is a U.S. National Stage Application under 35 U.S.C. §371 of PCT Application No. PCT/KR2018/012688, filed Oct. 25, 2018, whichclaims priority to Korean Patent Application Nos. 10-2017-0139495, filedOct. 25, 2017, and 10-2018-0017204, filed Feb. 12, 2018, whose entiredisclosures are hereby incorporated by reference.

TECHNICAL FIELD

The present disclosure relates to a mobile robot and a method ofcontrolling the same, and more particularly, to a mobile robot fortraveling a cleaning region, learning an obstacle, and performingcleaning to avoid an obstacle.

BACKGROUND ART

In general, a mobile robot is an apparatus for automatic cleaning byabsorbing a foreign substance such as dust from a floor surface whileautonomously traveling an area as a cleaning target without manipulationof a user.

Such a mobile robot detects a distance to an obstacle such as furniture,office supplies, a wall, or the like, which is installed within acleaning area and performs an obstacle avoidance operation.

A mobile robot includes an obstacle detection device such as infraredrays or laser beams as a device for detecting an obstacle. The mobilerobot may detect and avoid an obstacle within a predetermined distancebased on a detection signal by a sensor such as infrared rays or laserbeams.

When an obstacle is detected, if a path is immediately changed, themobile robot may not be capable of cleaning a corresponding region, andthus, may change the path after approaching the obstacle to the maximumor colliding with the obstacle.

However, when the mobile robot approaches the obstacle for cleaning andcollides with the obstacle, there may be a problem in that a targetobject is damaged due to collision with the mobile robot. For example,there may be a problem in that a pot on a table falls, the obstacle isdamaged, or a position of the obstacle is changed.

The mobile robot slips while colliding with the obstacle, and thus,there is a problem in that an error occurs between an actual positionand a position determined by the mobile robot.

Thus, it is required to recognize a type of an obstacle and to operatethe mobile robot depending on the obstacle.

Korean Patent Publication No. 10-2016-01575 discloses a robot cleanerfor recognizing information such as the attributes of the obstacle usingimage information. Conventionally, in the case of a human foot, aseparate recognition module is further configured to easily recognizethe human foot. However, conventionally, an obstacle is recognized afterthe robot cleaner approaches the obstacle, and thus, there is a problemin that a response to the obstacle is actually delayed.

In addition, the mobile robot is basically designed to travel to avoidan obstacle, but there is a limit in recognizing any type of anobstacle. Accordingly, traveling of the mobile robot is impeded, andseriously, the mobile robot is confined not to travel any longer.

Thus, recently, test is performed through various experiments before aproduct is released, and thus, information on an obstacle is previouslylearned. However, it is not possible to accumulate information on allobstacles through prior learning due to a restrictive test environment,and thus, a mobile robot has a limit in responding to an obstacle.

CITED REFERENCE Patent Document

Korean Patent Publication No. 10-2016-01575

Technical Problem

An objective of the present disclosure is to provide a mobile robot anda method of controlling the same for avoiding an obstacle variousresponse motions and escaping from a dangerous situation by capturing animage while traveling, analyzing the feature of the obstacle through aplurality of captured images, and acquiring information on a newobstacle.

Technical Solution

According to the present disclosure, a mobile robot includes a moveablebody, an image acquirer configured to capture an image in a travelingdirection, an obstacle detection unit configured to detect an obstaclepositioned at a side toward the traveling direction, and a controllerconfigured to store a plurality of image data captured through the imageacquirer, to start a response motion at a predetermined time point atwhich the obstacle detection unit determines that the obstacle ispositioned within a predetermined distance, and to determine theresponse motion depending on the determined obstacle based on the imagedata acquired prior to the predetermined time point.

The controller may set a plurality of response motions to be performedaccording to a predetermined type of a detection signal input from theobstacle detection unit, and may select any one of the plurality ofresponse motions based on the image data.

The controller may determine whether the obstacle is positioned within apredetermined distance based on a distance from the obstacle, calculatedthrough the detection signal.

The controller may determine the obstacle based on image data capturedin the same traveling direction as a traveling direction of the body ata time point of determining that the obstacle is positioned at apredetermined distance.

The mobile robot may transmit the image data to a terminal or a serverand may receive information on the obstacle, included in the image data.

According to the present disclosure, a mobile robot may include amoveable body, an image acquirer configured to capture an image in atraveling direction, an obstacle detection unit configured to detect anobstacle positioned at a side toward the traveling direction, and acontroller configured to determine whether the body is confined due tothe obstacle in response to a traveling state, and to enable the mobilerobot to escape from a confinement situation in response to informationon the obstacle, acquired from the at least one image data capturedprior to the time point of determining confinement.

The mobile robot may include a controller configured to determinewhether the body is confined due to the obstacle in response to atraveling state, may transmit the at least one image data capturedbefore a time point of confinement to a server, may set a responsemotion to prevent the body from being confined in response to obstacleinformation received from the server, and may avoid the obstacle.

The present disclosure may include a mobile robot configured to analyzethe image data based on the pre-stored obstacle information, todetermine a type of the obstacle positioned at a side in a travelingdirection, and to perform any one of a plurality of response motions onthe obstacle at a time point at which the obstacle detection unitdetermines that the obstacle is positioned within a predetermineddistance, and a server configured to analyze image data received fromthe mobile robot and to acquire obstacle information.

When a confinement situation occurs, the controller may transmit atleast one image data captured prior to a predetermined time from a timepoint of determining confinement of the body or during traveling for aprevious predetermined distance to the server.

When the obstacle is an obstacle as a reason for confinement, thecontroller may perform any one of a plurality of response motions toavoid the obstacle.

The controller may perform any one response motion to output warning forconfinement among a plurality of response motions when the obstacle isincluded in a candidate for causing a confinement situation.

The server may extract the feature of the obstacle as the reason forconfinement of the mobile robot, may generate a recognition model forthe obstacle, and may generate obstacle information including a responsemotion for avoiding the obstacle or a response motion for escaping froma confinement situation.

The server may set a candidate for causing confinement with respect to adifferent obstacle with a similar shape to the obstacle in response tothe feature of the obstacle and may generate the obstacle information.

According to the present disclosure, a method of controlling a mobilerobot includes, while traveling, capturing an image in a travelingdirection and storing image data by an image acquirer, determining thatan obstacle is positioned within a predetermined distance through anobstacle detection unit, determining a response motion depending on thedetermined obstacle based on the image data acquired prior to apredetermined time point of determining that the obstacle is positionedwithin a predetermined distance, starting a response motion on theobstacle at the predetermined time point, and operating based on theresponse motion and traveling to avoid and pass through the obstacle.

According to the present disclosure, a method of controlling a mobilerobot includes periodically capturing an image in a traveling directionand storing image data through an image acquirer while traveling,determining whether a body is confined due to the obstacle in responseto a traveling state, when determining that the body is confined,acquiring obstacle information from at least one image data capturedprior to a time point of determining confinement, and escaping from aconfinement situation in response to the obstacle information.

The method may further include, when determining that the body isconfined, transmitting at least one image data captured prior to a timepoint of confinement to a server, and setting a response motion toprevent the body from being confinement of the body in response toobstacle information received from the server and to avoid the obstacle.

Advantageous Effects

A mobile robot and a method of controlling the same according to thepresent disclosure may capture an image while traveling, may analyze apre-captured image to set a plurality of response motions with respectto an obstacle, and may perform any one of the plurality of responsemotions, and thus, may perform a predetermined operation depending on anobstacle.

According to the present disclosure, even if the same detection signalis input when a plurality of different obstacles is detected, anobstacle may be determined through an image and different operations maybe performed depending on the obstacles.

According to the present disclosure, an image in a traveling directionmay be captured while traveling, and thus, an obstacle may be determinedthrough a pre-captured image predetermined point at which the obstacleis positioned within a predetermined distance.

According to the present disclosure, an obstacle may be pre-determinedthrough an image before the obstacle is positioned within apredetermined distance, and thus, a response operation may beimmediately performed at a time point at which the obstacle ispositioned within the predetermined distance.

According to the present disclosure, whether a mobile robot is confinedmay be determined depending on a traveling state, at least onepre-captured image before the mobile robot is confined may be analyzed,and an obstacle as the reason for confinement may be extracted.

According to the present disclosure, a response motion of escaping froma confinement situation may be set in response to an obstacle as thereason for confinement, thereby overcoming the confinement situation.

According to the present disclosure, a response motion of avoiding anobstacle as the reason for confinement may be set, thereby preventingconfinement in next traveling.

According to the present disclosure, a candidate for causing aconfinement situation may be set with respect to another obstacle havingthe same or similar shape to an obstacle as the reason for confinement,and a response motion may be set for another obstacle, therebypreventing confinement from occurring.

According to the present disclosure, an obstacle may be effectivelyavoided, an operation may be performed depending on a type of theobstacle, a mobile robot may be prevented from being damaged due tocollision with the obstacle, a position error due to collision with theobstacle may be prevented, an environment change in a traveling may bedetected, and the mobile robot may be operated.

According to the present disclosure, the mobile robot may performvarious responses to an obstacle and may avoid a dangerous situation byacquiring information on a new obstacle through analysis and learning ofan obstacle through a server and updating information on an existingobstacle.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a perspective view of a mobile robot according to an exemplaryembodiment of the present disclosure.

FIG. 2 is a diagram showing a horizontal viewing angle of the mobilerobot of FIG. 1 .

FIG. 3 is a front view of the mobile robot of FIG. 1 .

FIG. 4 is a diagram showing a bottom surface of the mobile robot of FIG.1 .

FIG. 5 is a schematic diagram of a mobile robot and a system forrecognizing according to an embodiment of the present disclosure.

FIG. 6 is a block diagram showing main components of the mobile robot ofFIG. 1 .

FIGS. 7 and 8 are diagrams showing an example of driving of a mobilerobot and an image captured while traveling according to the presentdisclosure.

FIG. 9 is a diagram for explaining recognition of a detection signal ofan obstacle and recognition of the obstacle in a mobile robot accordingto an embodiment of the present disclosure.

FIG. 10 is a diagram for explaining an operation of a mobile robotdepending on a type of an obstacle according to an embodiment of thepresent disclosure.

FIG. 11 is a diagram for explaining a mobile robot and a signal flowbetween devices for obstacle recognition according to an embodiment ofthe present disclosure.

FIG. 12 is a flowchart for explaining detection of an obstacle and aresponse motion in a mobile robot according to an embodiment of thepresent disclosure.

FIG. 13 is a flowchart for explaining recognition of an obstacle and acontrol method based thereon in a mobile robot according to a firstembodiment of the present disclosure.

FIG. 14 is a diagram for explaining image transmission for recognitionof an obstacle of a mobile robot according to a second embodiment of thepresent disclosure.

FIG. 15 is a diagram for explaining a method of determining aconfinement situation through an image of a mobile robot according tothe second embodiment of the present disclosure.

FIG. 16 is a diagram for explaining a method of controlling a mobilerobot according to the second embodiment of the present disclosure.

FIG. 17 is a flowchart for explaining a method of determining error of amobile robot according to the second embodiment of the presentdisclosure.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

The attached drawings for illustrating exemplary embodiments of thepresent disclosure are referred to in order to gain a sufficientunderstanding of the present disclosure, the merits thereof, and theobjectives accomplished by the implementation of the present disclosure.The invention may, however, be embodied in many different forms andshould not be construed as being limited to the embodiments set forthherein, rather, these embodiments are provided so that this disclosurewill be thorough and complete, and will fully convey the concept of theinvention to one of ordinary skill in the art. Meanwhile, theterminology used herein is for the purpose of describing particularembodiments and is not intended to limit the invention. Like referencenumerals in the drawings denote like elements. A control componentaccording to the present disclosure may include at least one processor.

FIG. 1 is a perspective view of a mobile robot according to an exemplaryembodiment of the present disclosure. FIG. 2 is a diagram showing ahorizontal viewing angle of the mobile robot of FIG. 1 . FIG. 3 is afront view of the mobile robot of FIG. 1 . FIG. 4 is a diagram showing abottom surface of the mobile robot of FIG. 1 .

Referring to FIGS. 1 to 4 , a mobile robot 1 according to an exemplaryembodiment of the present disclosure may be moved along a floor of acleaning area, and may include a body 10 for absorbing foreignsubstances such as dust on the floor and an obstacle detection devicedisposed on a front surface of the body 10.

The body 10 may include a casing 11 that forms an outer appearance andforms a space for accommodating therein components included in the body10, an absorption unit 34 that is disposed in the casing 11 and absorbsforeign substances such as dust or waste, and a left wheel 36(L) and aright wheel 36(R) that are rotatably installed in the casing 11. As theleft wheel 36(L) and the right wheel 36(R) are rotated, the body 10 maybe moved along the floor of the cleaning area, and during thisprocedure, foreign substances may be absorbed through the absorptionunit 34.

The absorption unit 34 may include an absorption fan (not shown) forgenerating absorption force, and an absorption inlet 10 h for absorbingair current generated via rotation of an absorption fan. The absorptionunit 34 may include a filter (not shown) for collecting foreignsubstances in the air current absorbed through the absorption inlet 10 hand a foreign substances collection container (not shown) in whichforeign substances collected by the filter are accumulated.

The body 10 may include a driving driver for driving the left wheel36(L) and the right wheel 36(R). The driving driver may include at leastdriving motor. At least one driving motor may include a left wheeldriving motor for rotation of the left wheel 36(L) and a right wheeldriving motor for rotation of the right wheel 36(R).

An operation of the left wheel driving motor and the right wheel drivingmotor may be independently controlled by a driving controller of acontroller, and thus, the body 10 may be moved forward, may be movedbackward, or may turn. For example, when the body 10 is moved forward,the left wheel driving motor and the right wheel driving motor may berotated in the same direction, but when the left wheel driving motor andthe right wheel driving motor are rotated at different speeds or arerotated in different directions, a driving direction of the body 10 maybe changed. The body 10 may further include at least one auxiliary wheel37 for stably supporting the body 10.

The body 10 may further include a plurality of brushes 35 that ispositioned at a front side of a bottom surface unit of the casing 11 andhas brushes including a plurality of wings that radially extend. Throughrotation of the plurality of brushes 35, dusts may be removed from afloor of a cleaning area, and the dusts collected from the floor may beabsorbed through the absorption inlet 10 h and may be collected in acollection container.

The body 10 may further include a control panel that is disposed on anupper surface of the casing 11 and receives various commands for controlof the mobile robot 1 from a user.

As shown in FIG. 1A, the detection device may include a sensing unit 150for detecting an obstacle using a plurality of sensors and an imageacquirer 170 for capturing an image.

As shown in FIG. 1B, the detection device may be disposed on a frontsurface of the body 10 and may include an obstacle detector 100 fordetecting an obstacle and the image acquirer 170 for capturing an image.The obstacle detector 100 may detect an obstacle through an imagecaptured by emitting an optical pattern. The obstacle detector 100 mayinclude a pattern acquirer 140 and may also include the sensing unit150.

The image acquirer 170 may be configured to face forward and tophotograph a side toward a traveling direction or, as necessary, may beconfigured to face a ceiling. When two image acquirers are configured,the image acquirers may be installed on a front surface and an upper endportion of the body to capture images of a front side and a ceiling,respectively.

The obstacle detector 100 may be disposed on a front surface of the body10.

The obstacle detector 100 may be fixed to the front surface of thecasing 11 and may include a first pattern emission unit 120, a secondpattern emission unit 130, and the pattern acquirer 140. In this case,the pattern acquirer 140 may be disposed below the pattern emissionunits or may be disposed between the first and second pattern emissionunits as shown in the drawing.

An emission angle θh indicated in FIG. 2 may indicate a horizontalemission angle of first pattern beam P1 emitted from the first patternemission unit 120, may indicate an angle between the first patternemission unit 120 and opposite ends of a horizontal line, and may bedetermined in a range of 130° to 140°, but the present disclosure is notlimited thereto. A dotted line indicated in FIG. 2 is directed toward afront side of the mobile robot 1, and the first pattern beam P1 may beconfigured to be symmetric with the dotted line.

The body 10 may include a rechargeable battery 38, and a chargingterminal 33 of the battery 38 may be connected to a commercial powersource (e.g., a power socket in the home) or the body 10 may be dockedon the separate charging stand 400 connected to the commercial powersource and may be connected to the commercial power source to rechargethe battery 38. Electrical components included in the mobile robot 1 mayreceive power from the battery 38, and accordingly, when the battery 38is recharged, the mobile robot 1 may autonomously drive in a state inwhich the mobile robot 1 is electrically separated from the commercialpower source.

FIG. 5 is a schematic diagram of a mobile robot and a system forrecognizing according to an embodiment of the present disclosure.

The mobile robot 1 may receive a cleaning command with respect to apredetermined cleaning region of a traveling region H and may performcleaning. The mobile robot 1 may analyze an image captured whiletraveling and may determine a type of an obstacle.

The mobile robot 1 may detect a plurality of obstacles 2 and 3 presentin the traveling region H to recognize an obstacle and may perform anyone of approaching, passing, and avoiding operations depending on a typeof the obstacle. In this case, the mobile robot 1 may avoid an obstaclerather than approaching the obstacle, may approach the obstacle at apredetermined distance and may then preform a predetermined operation,may approach the obstacle and may then avoid the obstacle, or may passthrough the obstacle depending on a shape of the obstacle.

The mobile robot 1 may photograph a side toward a traveling directionthrough the image acquirer 170 included in the mobile robot 1 whiletraveling and may detect an obstacle positioned within a predetermineddistance through the obstacle detector 100. The mobile robot 1 maycontinuously photograph a side toward a traveling direction through theimage acquirer 170 or may capture an image at a predetermined intervalaccording to setting.

The mobile robot 1 may store an image captured while traveling, and whendetecting an obstacle through the obstacle detector 100, the mobilerobot 1 may select performable response motions corresponding thereto,and may then determine and perform a response motion corresponding tothe obstacle depending on the obstacle, in particular, depending on atype of the obstacle that is determined through image analysis.

Even if the same detection signal is input with respect to differentobstacles, the mobile robot 1 may analyze an obstacle based on acaptured image, may set a response motion corresponding to the obstacle,and accordingly, may perform an operation corresponding thereto.

When capturing an image, the mobile robot 1 may analyze the image at apredetermined interval and may recognize an obstacle included in theimage, and when the obstacle detector 100 detects the obstacle to bepositioned within a predetermined distance, the mobile robot 1 mayperform an operation corresponding to the response motion based on thepredetermined type of the obstacle.

For example, when the determined obstacle is a dangerous obstacle, themobile robot 1 may avoid the obstacle rather than approaching theobstacle, and when the determined obstacle is a general obstacle, themobile robot 1 may approach the obstacle at a close distance or maycollide with the obstacle and may then avoid the obstacle.

The mobile robot 1 may determine a type of an obstacle based on datastored therein and may be connected to a server 90 or a terminal 80 andmay determine the type of the obstacle. The mobile robot 1 may beconnected to a separate communication device (not shown) in order tocommunicate with the server 90 through a network N.

The mobile robot 1 may filter each image data with respect to an imagecaptured while traveling and may transmit the image data to a server orthe terminal at a predetermined interval to make a request for obstacleinformation, and thus, may receive the obstacle information included inthe image data from the server or the terminal. When the obstacledetector 100 determines the obstacle to be positioned within apredetermined distance, the mobile robot 1 may immediately check theobstacle through the image data.

The terminal 80 may include an application for controlling the mobilerobot 1, may display a map of a traveling zone to be cleaned by themobile robot 1 by executing the application, and may determine aspecific region to make the mobile robot 1 clean the region on the map.The terminal 80 may display a position of the mobile robot based on datareceived from the mobile robot and may display information on a cleaningstate.

The terminal may be a device that includes a communication moduleinstalled therein, is connected to a network, and has a program forcontrolling the mobile robot or an application for controlling themobile robot installed therein, and may be, a computer, a laptop, asmart phone, a PDA, or a tablet PC. In addition, the terminal may alsobe a wearable device such as a smart watch.

The terminal 80 may be connected to the mobile robot through a networkestablished in a home. The terminal and the mobile robot may beconnected using WIFI and may also communicate with each other using ashort distance wireless communication method such as Bluetooth, infraredray communication, or Zigbee. It may be noted that a communication ofthe terminal and the mobile robot is not limited thereto.

The terminal 80 may determine a type of an obstacle in response to animage received from the mobile robot 1. The terminal 80 may determinethe type of the obstacle based on pre-stored data or may be connected toa server to determine the type of the obstacle. For example, when it isnot possible to connect the mobile robot 1 to the server via connectionwith a network, the terminal 80 may receive data of the mobile robot andmay transmit the data to the server.

The terminal 80 may display an image on a screen, may input a type of anobstacle through user input, may transmit the input information to themobile robot, and may determine an operation of a specific obstaclethrough a menu of an application.

The server 90 may analyze image data received from the mobile robot 1connected thereto through a predetermined network N, may determine atype of an obstacle, and may transmit a response thereto to the mobilerobot 1. When there is a request from the terminal, the server 90 maydetermine the type of the obstacle and may respond to the request.

The server 90 may accumulate image data received from the plurality ofmobile robots 1, may analyze the plurality of image data, and may learnthe obstacle.

The server 90 may include a database (not shown) for recognizing anobstacle based on the plurality of image data, and may recognize thefeature of the obstacle, extracted from the image data, and maydetermine a type of the obstacle.

The server 90 may accumulate image data received from the plurality ofmobile robots 1, may analyze the plurality of image data, and may learnthe obstacle.

The server 90 may include a database (not shown) for recognizing anobstacle based on the plurality of image data, and may recognize thefeature of the obstacle, extracted from the image data, and maydetermine a type of the obstacle.

The server 90 may accumulate and store obstacle information, may analyzethe obstacle information, and may determine a type of the obstacle. Theserver 90 may classify the obstacle depending on a type thereof, and mayset an operation of the mobile robot 1 with respect to the obstacle asat least one response motion.

The server 90 may analyze new obstacle information and may updatepre-stored obstacle informant. The server 90 may receive and storeinformation on an operation of the mobile robot with respect to anobstacle, which is set or changed by the terminal 80, may match theinformation with the pre-stored obstacle information, and may updatesetting of an operation of the mobile robot with respect to theobstacle.

FIG. 6 is a block diagram showing main components of the mobile robot ofFIG. 1 .

As shown in FIG. 6 , the mobile robot 1 may include the obstacledetector 100, the image acquirer 170, a cleaner 260, a traveling unit250, a data unit 180, an output unit 190, a manipulator 160, acommunicator 280, and a controller 110 for controlling an overalloperation.

The manipulator 160 may include at least one button, a switch, and atouch input device, and may receive on/off or various commands requiredfor an overall operation of the mobile robot 1 and may input thecommands to the controller 110.

The output unit 190 may include a display such as an LED or an LCD andmay display an operation mode, reservation information, a battery state,an operation state, an error state, or the like. The output unit 190 mayinclude a speaker or a buzzer and may output a predetermined soundeffect, a warning horn, or voice guidance corresponding to an operationmode, reservation information, a battery state, an operation state, oran error state.

The communicator 280 may communicate with the terminal 80 using awireless communication method. The communicator 280 may be connected tothe Internet and may communicate with the external server 90 through anetwork in a home.

The communicator 280 may transmit a generated map to the terminal 80,may receive a cleaning command from the terminal, and may transmit dataof an operation state and a cleaning state of the mobile robot to theterminal. The communicator 280 may transmit and receive data using acommunication module such as WiFi or Wibro as well as short distancewireless communication such as Zigbee or Bluetooth.

The communicator 280 may transmit information on an obstacle detectedfrom the obstacle detector 100 to the server 90 through the communicator280 and may receive data of the obstacle from the server. Thecommunicator 280 may receive information on an obstacle present in atraveling zone and operation information based thereon from the terminal80 and may transmit operation data of the mobile robot to the terminal80.

The traveling unit 250 may include at least one driving motor and mayenable the mobile robot to travel according to a control command of atraveling controller 113. As described above, the traveling unit 250 mayinclude a left wheel driving motor for rotating the left wheel 36(L) anda right wheel driving motor for rotating the right wheel 36(R).

The cleaner 260 may operate a brush to make a current state to a statein which dusts or foreign substances around the mobile robot are easilyabsorbed and may operate an absorption device to absorb dusts or foreignsubstances. The cleaner 260 may control an operation of an absorptionfan included in the absorption unit 34 for absorbing a foreign substancesuch as dusts or wastes to inject dusts to a foreign substancescollection container (dust container) through an absorption inlet.

The cleaner 260 may further include a mop cleaner (not shown) that isinstalled behind a bottom surface of the body to contact the floor andis configured to damp mop the floor, and a water bottle (not shown)configured to supply water to the mop cleaner.

A battery (not shown) may supply power required for an overall operationof the mobile robot 1 as well as the driving motor. When the battery isdischarged, the mobile robot 1 may travel to return to a charging stand400 for charging, and while traveling to return, the mobile robot 1 mayautonomously detect a position of the charging stand. The charging stand400 may include a signal transmitter (not shown) for transmitting apredetermined return signal. The return signal may be an ultrasonicsignal or an infrared (IR) signal, but is not limited thereto.

The data unit 180 may store a detection signal input from the obstacledetector 100 or the sensing unit 150, may store reference data fordetermining an obstacle, and may store obstacle information on thedetected obstacle.

The data unit 180 may store obstacle data 181 for determining a type ofan obstacle, image data 182 for storing a captured image, and map data183 of a region. The map data 183 may include obstacle information, andmay store a basic map of a region which is searched by the mobile robotand in which the mobile robot is capable of traveling, a cleaning mapformed by dividing the basic map into regions, a user map generated byorganizing shapes of regions of the cleaning map to make a user checkthe same, and a guide map displayed by overlapping the cleaning map andthe user map with each other.

The obstacle data 181 may be data for recognizing an obstacle anddetermining a type of the obstacle and may include information on anoperation of a mobile robot with respect to the recognized obstacle, forexample, motion information on a traveling speed, a traveling direction,whether a mobile robot avoids the obstacle, whether the mobile robotstops, or the like, and may include data of a sound effect, a warninghorn, and voice guidance output through a speaker 173. The image data182 may include a captured image and recognition information forobstacle recognition received from the server.

The data unit 180 may store control data for controlling an operation ofthe mobile robot, data of a cleaning mode of the mobile robot, and adetection signal of ultrasonic wave/laser or the like of the sensingunit 150.

The data unit 180 may store data to be read by a microprocessor and mayinclude a hard disk drive (HDD), a solid state disk (SSD), a silicondisk drive (SDD), ROM, RAM, CD-ROM, a magnetic tape, a floppy disk, oran optical data storage device.

The obstacle detector 100 may include the first pattern emission unit120, the second pattern emission unit 130, and the pattern acquirer 140.As described above with reference to FIG. 1A, when the pattern emissionunit is not included, an obstacle may be detected from an image capturedby an image acquirer without an emitted pattern.

The obstacle detector 100 may include a plurality of image acquirers.When the obstacle detector 100 includes a pattern emission unit, theobstacle detector 100 may further include an image acquirer forcapturing an image including a pattern and a separate image acquirer forphotographing a side in a traveling direction or a forward upper side.

The obstacle detector 100 may include the first pattern emission unit120, the second pattern emission unit 130, and the pattern acquirer 140.The obstacle detector 100 may include the sensing unit 150 included inat least one sensor. As necessary, the obstacle detector 100 may includea sensor.

As described above, the obstacle detector 100 may be installed on afront surface of the body 10, may emit the first and second pattern beamP1 and P2 to a front side of the mobile robot, and may photographemitted pattern beam to acquire an image including the pattern. Theobstacle detector 100 may input the acquired image to the controller 110as the obstacle detection signal.

The first and second pattern emission units 120 and 130 of the obstacledetector 100 may include a light source and an optical patternprojection element (OPPE) for generating a predetermined pattern byprojecting light emitted from the light source. The light source may bea laser diode (LD), a light emitting diode (LED), or the like. The LDmay be capable of precisely measuring a distance due to excellentmonochromaticity, straight characteristics, and interfacecharacteristics compared with other light sources, and in particular,infrared rays or visible rays have a problem in that a large deviationoccurs in accuracy in measuring a distance for the reason such as coloror material of a target object, and thus, the LD may be used as a lightsource. An optical pattern projection element (OPPE) may include a lensor a diffractive optical element (DOE). Beams with various patterns maybe emitted depending on a configuration of an OPPE included in each ofthe pattern emission units 120 and 130.

The pattern acquirer 140 may acquire an image of a front side of thebody 10. In particular, the pattern beams P1 and P2 may be indicated inan image (hereinafter, an acquired image) acquired by the patternacquirer 140, and hereinafter, an image of the pattern beams P1 and P2indicated on the acquired image may be referred to as a light pattern,and the light pattern is an image formed by focusing the pattern beamsP1 and P2 that is substantially incident on an actual space on an imagesensor, and thus, reference numerals such as the pattern beams P1 and P2are denoted, and images corresponding to the first pattern beam P1 andthe second light pattern P2, respectively, may be referred to as thefirst light pattern P1 and the second light pattern P2, respectively.

When the pattern emission unit is not included, the pattern acquirer 140may acquire an image of a front side of the body, which does not includea pattern beam.

The pattern acquirer 140 may include a camera for converting an image ofa subject into an electric signal, re-changing the electric signal to adigital signal, and then recording the digital signal in a memorydevice. The camera may include an image sensor (e.g., a CMOS imagesensor) including at least one optical lens and a plurality of didoes(photodiodes, e.g., pixels) for forming an image by light passingthrough the optical lens, and a digital signal processor (DSP) forconfiguring an image based on a signal output from the photodiodes. TheDSP may generate a video image including frames including a still imageas well as a still image.

The image sensor may be a device for converting an optical image into anelectric signal and may include a chip having a plurality of opticaldiodes integrated thereon, and the optical diode may be, for example, apixel. Electric charges may be accumulated on each pixel according to animage formed on a chip by light passing through the lens, and theelectric charges accumulated on the pixel may be converted into anelectric signal (e.g., a voltage). A charge coupled device (CCD), acomplementary metal oxide semiconductor (CMOS), or the like has beenwell known as an image sensor.

An image processor may generate a digital image based on an analogsignal output from an image sensor. The image processor may include anAD converter for converting an analog signal into a digital signal, abuffer memory for temporally recording digital data according to thedigital signal output from the AD converter, and a digital signalprocessor (DSP) for processing the information recorded in the buffermemory to configure a digital image.

The obstacle detector 100 may analyze a pattern through the acquiredimage and may detect an obstacle depending on a shape of the pattern,and the sensing unit 150 may detect an obstacle positioned at adetection distance of each sensor through a sensor.

The sensing unit 150 may include a plurality of sensors to detect anobstacle. The sensing unit 150 may detect a front side of the body 10,that is, an obstacle in a traveling direction using at least one of alaser beam, an ultrasonic wave, or infrared rays. The sensing unit 150may further include a cliff detection sensor for detecting whether acliff is present on the floor in a traveling zone. When a transmittedsignal is reflected and input to the sensing unit 150, the sensing unit150 may input information on whether an obstacle is present orinformation on a distance to the obstacle to the controller 110 as anobstacle detection signal.

When the mobile robot is operated, the image acquirer 170 may capturecontinuous images. The image acquirer 170 may capture an image with apredetermined period. The image acquirer 170 may capture an image evenin a traveling or cleaning state in which an obstacle is not detected bythe obstacle detector 100.

For example, after the image acquirer 170 performs photography once,when the mobile robot is moved in a traveling direction, the size of aphotographed obstacle is just changed in an image captured once whilethe traveling direction is maintained rather than being changed, andthus, the image may be periodically captured. The image acquirer 170 maycapture an image in units of predetermined time or predetermineddistance. The image acquirer 170 may capture a new image while thetraveling direction is changed.

The image acquirer 170 may set a photography period depending on amoving speed of the mobile robot. The image acquirer 170 may set thephotography period in consideration of a detection distance of thesensor unit and the moving speed of the mobile robot.

The image acquirer 170 may store an image captured while the bodytravels in the data unit 180 as the image data 182.

The obstacle detector 100 may detect an obstacle positioned in a path ina traveling direction and may input to the detection signal to acontroller. The obstacle detector 100 may input information on aposition of the detected obstacle or information on movement thereof tothe controller 110. The pattern acquirer 140 may include an imageincluding the pattern emitted by the pattern emission unit to thecontroller as a detection signal, and the sensing unit 150 may input adetection signal of an obstacle detected by a sensor to the controller.

The controller 110 may control the traveling unit 250 to allow themobile robot to travel in a traveling zone determined in the travelingregion H.

While traveling, the controller 110 may control the traveling unit 250and the cleaner 260 to absorb dusts or foreign substances around themobile robot, and thus, may perform cleaning in the traveling zone.Thus, the cleaner 260 may operate a brush to make a current state to astate in which dusts or foreign substances around the mobile robot areeasily absorbed and may operate an absorption device to absorb dusts orforeign substances. The controller 110 may control the cleaner to absorbforeign substance and to perform cleaning while traveling.

The controller 110 may check charge capacity of a battery and maydetermine a time of returning to a charging stand. When the chargingcapacity is reached to a predetermined value, the controller 110 maystop a performed operation, and may start searching for the chargingstand in order to return to the charging stand. The controller 110 mayoutput notification of the charge capacity of the battery andnotification of return to the charging stand.

The controller 110 may set an operation mode of the mobile robot byprocessing input data according to manipulation of the manipulator 160,may output an operation state through the output unit 190, and mayoutput a warning horn, sound effect, or voice guidance based ondetection of an operation state, an error state, or an obstacle througha speaker.

The controller 110 may recognize an obstacle detected by the imageacquirer 170 or the obstacle detector 100 and may set and perform anyone of a plurality of response motions, which corresponds to theobstacle.

The controller 110 may determine the obstacle from an image captured bythe image acquirer 170, and when the obstacle detector 100 detects theobstacle to be positioned within a predetermined distance, thecontroller may set and operate the response motion with respect to theobstacle.

Before the obstacle detector 100 detects the obstacle, the controller110 may analyze an image captured through the image acquirer 170, i.e.,image data and may determine an obstacle included in the image data. Thecontroller 110 may determine an obstacle through data contained therein,may also transmit image data to a server or a terminal through acommunicator and may determine a type of the obstacle.

When the obstacle detector 100 detects the obstacle to be positionedwithin a predetermined distance after an image is captured, if a sensorunit for setting a response motion depending on a type of acorresponding obstacle detects the corresponding obstacle based on adetection signal, the controller 110 may make the body perform adetermined operation depending on a type of pre-recognized obstacle.

The controller 110 may perform an operation corresponding to an obstaclewhen the obstacle is detected depending on a detection distance of anobstacle detection unit, and even if the obstacle is detected by theobstacle detection unit, when the obstacle is positioned within apredetermined distance, an operation corresponding to the obstacle maybe performed.

For example, when a distance at which an obstacle is initially detectedfrom a detection signal by a pattern acquirer and a distance at whichthe obstacle is detected by an ultrasonic sensor, a time of inputting adetection signal with respect to an obstacle may be changed. Thus, whenthere is a plurality of elements for detecting an obstacle or a distancefor detecting an obstacle is equal to or greater than a predetermineddistance like in a laser sensor, the controller 110 may perform any oneof a plurality of response motions based on a distance to an obstacledetected by the obstacle detector 100 when the obstacle is positioned ata predetermined distance, for example, 30 cm.

The controller 110 may determine performable response motions among aplurality of response motions depending on a shape of a detection signalbased on a detection signal with respect to an obstacle, may analyzeimage data to recognize an obstacle included in the image, andaccordingly, may select any one of the response motion to control anoperation of the body.

For example, when the detection signal corresponds to an image includedin a pattern, the controller may determine an obstacle based ondifferent detection signals depending on a shape of a pattern and mayidentify the obstacle according to a detection signal, that is, theshape of the pattern.

When recognizing an obstacle through an image, the controller 110 maydetermine the obstacle with respect to the captured with a predeterminedperiod before the obstacle detection unit inputs a detection signal orat a time when an obstacle detection signal is input. As necessary, thecontroller 110 may transmit data to the server 90 or the terminal 80 andmay receive data on an obstacle.

For example, when the controller 110 inputs the same detection signalwith respect to different obstacles, thresholds, fans, or tables, thecontroller 110 may select a plurality of performable response motionswith respect to the corresponding detection signal, set respectiveresponse motions with respect to thresholds, fans, or tables and maycontrol an operation through image data.

The controller 110 may include an obstacle recognition unit 111, a mapgenerator 112, and the traveling controller 113.

When an initial operation is performed or a map of a cleaning region isnot stored, the map generator 112 may generate a map of a cleaningregion based on obstacle information while the mobile robot travels inthe cleaning region. The map generator 112 may generate a pre-generatedmap based on the obstacle information acquired while traveling.

The map generator 112 may generate a basic map based on informationacquired from the obstacle recognition unit 111 while traveling and maygenerate a cleaning map by dividing the basic map into regions. The mapgenerator 112 may generate the user map and the guide map by organizingregions of the cleaning maps and setting the attributes of the region.The basic map may be a map that indicates a shape of a cleaning regionacquired through traveling using an outline thereof and the cleaning mapmay be a map formed by dividing the basic map into regions. The basicmap and the cleaning map may information on a region in which the mobilerobot is capable of traveling and obstacle information. The user map maybe a map formed by simplifying regions of the cleaning map, organizingand processing shapes of outlines of the regions, and adding a visualeffect. The guide map may be a map obtained by overlapping the cleaningmap and the user map with each other. The guide map indicates thecleaning map, and thus, a cleaning command may be input based on aregion in which the mobile robot actually travels.

After generating the basic map, the map generator 112 may generate a mapby dividing a cleaning region into a plurality of regions, adding aconnection path for connecting the plurality of regions to the map, andadding information on an obstacle in each region to the map. The mapgenerator 112 may generate a map in which regions are separated bydividing the region on the map into sub-regions to set a representativeregion, setting the divided sub-regions as separate detailed regions,and merging the detailed regions to the representative region.

The map generator 112 may process shapes of regions with respect to theseparated regions. The map generator 112 may set the attributes of theseparated regions, and may process the shapes of the regions dependingon the attributes of the regions.

The obstacle recognition unit 111 may determine an obstacle throughinput from the image acquirer 170 or the obstacle detector 100, and themap generator 112 may generate a map of a traveling zone and may addinformation on the detected obstacle to the map. The travelingcontroller 113 may control the traveling unit 250 to change a movingdirection or a traveling path in response to the obstacle information orto travel to pass through the obstacle or avoid the obstacle.

The traveling controller 113 may control the traveling unit 250 toindependently operate a left wheel driving motor and a right wheeldriving motor, and thus, the body 10 may travel to go straight or toturn. The traveling controller 113 may control the traveling unit 250and the cleaner 260 depending on a cleaning command and may absorb aforeign substance to perform cleaning while the body 10 travels in thecleaning region.

The obstacle recognition unit 111 may analyze data input from theobstacle detector 100 and may determine an obstacle. The obstaclerecognition unit 111 may calculate a distance of the obstacle or adistance to the obstacle according to a detection signal of the obstacledetection unit, for example, a signal of an ultrasonic wave or a laserbeam, may analyze an acquired image including a pattern to extract thepattern, and may determine the obstacle. When an ultrasonic wave orinfrared ray signal is used, a shape of a received ultrasonic wave and atime of receiving the ultrasonic wave are changed depending on adistance of the obstacle or a position of the obstacle, the obstaclerecognition unit 111 may determine an obstacle based thereon.

When the image acquirer 170 inputs image data obtained by photographingan obstacle, the obstacle recognition unit 111 may store the image datain the data unit 180. The image acquirer 170 may photograph a frontobstacle a plurality of number of times, and thus, a plurality of imagedata may also be stored. When the image acquirer 170 continuouslycaptures an image in a traveling direction, the obstacle recognitionunit 111 may store an input video image as image data or may divide thevideo image in frame units and may store the video image as image data.

The obstacle recognition unit 111 may analyze a video image in frameunits, may remove an unnecessary frame, that is, a frame in which atarget object shakes, an unfocused frame, or an empty frame (a frame inwhich an obstacle is not photographed), and may store a frame as imagedata in predetermined time units.

The obstacle recognition unit 111 may analyze a plurality of image dataand may determine whether a photographed target object, that is, anobstacle is recognized. In this case, the obstacle recognition unit 111may analyze image data and may determine whether image data isrecognizable. For example, the obstacle recognition unit 111 mayseparate and discard a shaking image, an unfocused image, or an image inwhich an obstacle is identified due to darkness.

The obstacle recognition unit 111 may analyze image data to extract thefeature of the obstacle and may determine based on the shape, size, andcolor of an obstacle to determine a position of the obstacle.

The obstacle recognition unit 111 may analyze an obstacle from aplurality of pre-captured images, and may analyze an image capturedprior to a predetermined time based on a time of determining theobstacle to be positioned at a predetermined distance to determine theobstacle upon receiving a detection signal of the obstacle.

Only a portion of an obstacle is photographed in a state in which themobile robot approaches the obstacle within a predetermined distance,and thus, the obstacle may be determined using an image captured byphotographing an entire shape of the obstacle because the obstacle isphotographed prior to a predetermined time, that is, the obstacle isphotographed at a farther distance than a predetermined distance. Theobstacle recognition unit 111 may determine a detailed type of theobstacle and, as necessary, may determine only the shape and size of theobstacle.

The obstacle recognition unit 111 may exclude a background of an imagefrom image data, may extract the feature of the obstacle based onpre-stored obstacle data, and may determine a type of the obstacle. Theobstacle data 181 may be updated based on new obstacle data receivedfrom a server. The mobile robot 1 may store obstacle data of thedetected obstacle and may receive data of the type of the obstacle froma server with respect to other data.

The obstacle recognition unit 111 may detect feature such as a point, aline, or a surface from predetermined pixels included in an image andmay detect an obstacle based on the detected feature.

The obstacle recognition unit 111 may extract an outline of an obstacle,may recognize the obstacle based on a shape thereof, and may determine atype of the obstacle. The obstacle recognition unit 111 may determinethe type of the obstacle depending on the color or size of the obstaclebased on the shape. The obstacle recognition unit 111 may determine thetype of the obstacle based on the shape and movement of the obstacle.

The obstacle recognition unit 111 may differentiate the human, ananimal, and an object therebetween based on obstacle information. Theobstacle recognition unit 111 may classify a type of the obstacle into ageneral obstacle, a dangerous obstacle, a bio obstacle, and a floorobstacle and may determine a detailed type of the obstacle with respectto each classification.

The obstacle recognition unit 111 may transmit recognizable image datato the server 90 through the communicator 280 and may determine a typeof the obstacle. The communicator 280 may transmit at least one imagedata to the server 90.

When receiving image data from the mobile robot 1, the server 90 mayanalyze image data to extract an outline or shape of the photographedobject, and may compare the extracted information with pre-stored dataof the object, and may determine a type of the obstacle. The server 90may preferentially search for obstacles with a similar shape or asimilar color, may extract feature from corresponding image data, andmay compare the information, and thus, may determine a type of theobstacle.

The server 90 may determine a type of an obstacle and may then transmitdata of the obstacle to the mobile robot 1.

The obstacle recognition unit 111 may store data of an obstacle,received from a server through a communicator, in the data unit 180 asobstacle data. When the server determines the type of the obstacle, theobstacle recognition unit 111 may perform an operation correspondingthereto. The traveling controller 113 may control the traveling unit toavoid, approach, or pass through the obstacle in response to a type ofthe obstacle, and as necessary, may output a predetermined sound effector a warning horn, or voice guidance through a speaker.

As described above, the obstacle recognition unit 111 may determinewhether image data is recognizable and may transmit image data to theserver according to stored obstacle data, and thus, may recognize a typeof the obstacle according to response of the server.

The obstacle recognition unit may store obstacle data for obstaclerecognition with respect to a selected obstacle among a plurality ofobstacles, and thus, even if image data is not transmitted to theserver, the obstacle recognition unit may recognize the obstacle basedon obstacle recognition data.

There is a limit in a storage capacity of the data unit 180, and thus,the controller 110 may store information on a portion of the selectedobstacle as obstacle recognition data. For example, the controller 110may store obstacle recognition data in a data unit with respect to anobstacle selected through the terminal 80 or an obstacle that isdetected large numbers of times based on a detection number of times.

Thus, the obstacle recognition unit 111 may information on an obstaclepresent in a cleaning region or a repeatedly detected obstacle in thedata unit, and thus, when an obstacle is detected, an operationcorresponding thereto may be immediately performed.

When recognizing a type of an obstacle from image data, the travelingcontroller 113 may control the traveling unit 250 to allow the body 10to perform a predetermined operation in response to the type of theobstacle.

When determining the obstacle to be positioned within a predetermineddistance according to a detection signal of the obstacle detector 100,the traveling controller 113 may set and perform any one of a pluralityof response motions depending on a type, shape, or size of the obstacle,determined based on image data, with respect to a response motion to beexecuted depending on the type or shape of the detection signal.

The traveling controller 113 may determine whether the mobile robot iscapable of traveling or entering, may set a traveling path/cleaning pathto allow the mobile robot to approach the obstacle, to pass through theobstacle, or to avoid the obstacle, and may control the traveling unit250 in response to the obstacle recognized by the obstacle recognitionunit 111.

For example, the traveling controller 113 may stop, decelerate,accelerate, reverse, U-turn, and change a traveling direction inresponse to the obstacle, may prevent the body 10 from approach theobstacle at a predetermined distance or greater, and may make the body10 to stand by for a predetermined time. The traveling controller 113may output sound determined depending on an obstacle through a speakerand may output the sound with a predetermined operation.

When the obstacle detector 100 detects an obstacle to be positionedwithin a predetermined distance, the traveling controller 113 may set aplurality of response motions such as avoidance, approaching, setting ofan approaching distance, stoppage, deceleration, acceleration, reverse,U-turn, and a change in traveling direction depending on a detectionsignal, and may set any one response motion depending on an obstacledetermined from captured image data, may control the traveling unit.

That is, the traveling controller 113 may set a plurality of responsemotions depending on an input detection signal, and in this case, maycontrol the traveling unit to perform any one of a plurality of responsemotions in response to an obstacle determined from pre-captured imagedata prior to a time of inputting a detection signal.

When a detailed type of the obstacle is determined through image data,the traveling controller 113 may set a response motion depending on thetype of the obstacle, and even if it is not possible to accuratelydetermine the type of the obstacle, the traveling controller 113 may seta response motion depending on the shape or size of the obstacle. Forexample, when the detailed type of the obstacle is not known but a spacewith a predetermined size from a floor surface, that is, a space with aheight and a width for enabling the mobile robot to pass therethrough ispresent, the traveling controller 113 may set the response motion to thebody to pass through the obstacle.

Hereinafter, a mobile robot according to a second embodiment will bedescribed.

The mobile robot according to the second embodiment may be configured asshown in FIGS. 1 to 6 showing a first embodiment. The mobile robotaccording to the second embodiment may be configured with the samehardware as in the first embodiment, and a control component basedthereon is shown in FIG. 6 .

As described above with reference to FIG. 6 , the mobile robot 1according to the second embodiment may include the obstacle detector100, the image acquirer 170, the cleaner 260, the traveling unit 250,the data unit 180, the output unit 190, the manipulator 160, thecommunicator 280, and the controller 110 for controlling an overalloperation. The controller 110 may include the obstacle recognition unit111, the map generator 112, and the traveling controller 113.

Thus, with regard to the mobile robot according to the secondembodiment, the same term and the same reference numeral are denoted forthe same component as in the first embodiment. The mobile robotaccording to the second embodiment may comply with the description ofthe first embodiment with respect to the same components as in the firstembodiment, and a description thereof will be omitted below.

The mobile robot according to the second embodiment may capture aplurality of images while traveling, and in this case, may periodicallycapture and store an image, may pre-detect an obstacle through thecorresponding image, and may then perform a predetermined operation whena distance from the obstacle becomes a predetermined distance.

When image data obtained by photographing a side toward a travelingdirection is stored, the obstacle recognition unit 111 may determine anobstacle through the stored image data. Even if a separate obstacledetection signal is not input from the obstacle detection unit, theobstacle recognition unit 111 may analyze image data and may recognizean obstacle included in the image data.

When the obstacle recognition unit 11 determines an obstacle from imagedata and then the obstacle is positioned within a predetermined distanceaccording to a detection signal of the obstacle detector 100, thetraveling controller may set a response motion in response to theobstacle and thus may perform a response motion.

That is, according to the first and second embodiments, after anobstacle is detected, an image may be analyzed and whether the obstacleis recognized or whether the obstacle is pre-recognized from an imagemay be determined, and in this regard, the obstacle may be detected fromthe obstacle, and a response motion may be set and performed dependingon a type of the obstacle.

The traveling controller 113 may determine a dangerous situation withrespect to various cases that occur while traveling, may transmit, tothe server, image data that is pre-captured based on a time point atwhich the dangerous situation occurs, and may make a request forobstacle information in order to prevent or prepare for the dangeroussituation.

For example, when a confinement situation occurs due to a predeterminedobstacle after the mobile robot enters the obstacle, the travelingcontroller 113 may transmit, to the server, an image captured for apredetermined time prior to a time point of determining a confinementsituation or at least one piece of image data captured while the mobilerobot travels. When a traveling distance for a predetermined time isless than a predetermined distance, the traveling controller 113 maydetermine a current situation to be a confinement situation.

The confinement situation may refer to a state in which movement of thebody is confined and is limited because it is not possible to move thebody for a predetermined distance or greater due to an obstacle.

While traveling, when a dangerous situation occurs, for example, aspecific object falls on or ahead of the body, the traveling controller113 may transmit image data captured before the dangerous situationoccurs, to the server. For example, a vase or the like may fall due tocollision with an obstacle while traveling.

The traveling controller 113 may output error and, as necessary, mayoutput a predetermined warning horn or voice guidance.

When a dangerous situation occurs, the traveling controller 113 mayreceive obstacle information generated based on the image datatransmitted to the server, may update the obstacle information of a dataunit, may determine the corresponding obstacle to be an obstacle as thereason for a dangerous situation, and may then may enable the mobilerobot to travel and avoid the obstacle.

The server 90 may analyze a plurality of pieces of received image datawith respect to a predetermined time or a predetermined distance, maydetermine an obstacle as the reason for the dangerous situation, maygenerate a recognition model for the corresponding obstacle, may set aresponse motion not to approach or enter the obstacle, may updateobstacle information with respect to the set response motion, and maytransmit the updated obstacle information to the mobile robot 1.

The server 90 may determine an obstacle as the reason for a confinementsituation, may set obstacle information including the feature of acorresponding obstacle and a response motion for the feature to enablethe mobile robot to avoid the corresponding obstacle without enteringthe obstacle, and may transmit the obstacle information to the mobilerobot. When a vase falls in the case of collision with an obstacle, theserver 90 may analyze an image, may determine an obstacle put on thevase, may set a response motion to prevent the mobile robot fromapproaching the corresponding obstacle, and may generate obstacleinformation.

The traveling controller 113 may enable the mobile robot to travel andavoid the corresponding obstacle according to a response motion withrespect to a confinement situation received from the server. When aconfinement situation occurs in a detected obstacle, the travelingcontroller 113 may control the traveling unit to approach and then avoidthe corresponding obstacle at a predetermined distance or to change atraveling direction and avoid the obstacle immediately after detectingthe obstacle.

FIGS. 7 and 8 are diagrams showing an example of driving of a mobilerobot and an image captured while traveling according to the presentdisclosure.

As shown in FIG. 7 , while traveling, the mobile robot 1 may detect anobstacle positioned ahead of the body 10.

When the mobile robot 1 travels toward a window 04 in the travelingregion H as shown in FIG. 7A, an image shown in FIG. 7B may be captured.The captured image may include a plurality of obstacles O01 to O06positioned at a side toward a traveling direction.

The image acquirer 170 may be continuously captured or may be repeatedlycaptured with a predetermined period.

When the mobile robot 1 travels straight, as the mobile robot approachesan obstacle, the obstacle may be photographed in an enlarged form asshown in FIG. 8 .

In the captured image, as the body of the mobile robot 1 travels, anarea of an obstacle positioned at a side toward a traveling direction ofthe mobile robot may be increased, and an area of an obstacle positionedat an opposite side to the traveling direction of the mobile robot maybe reduced.

As shown in FIG. 8A, as the mobile robot travels, the mobile robot mayapproach second and third obstacles O02 and O03 among the plurality ofobstacles O01 to O06 positioned in a region. Thus, the image capturedthrough the image acquirer 170 may include an image formed byphotographing the second and third obstacles O02 and O03 as shown inFIG. 8B.

The obstacle recognition unit 111 may store an image captured at thesame positioned as in FIG. 7 and may detect and recognize the pluralityof obstacles O01 to O06 through the stored image data. As shown in FIG.8 , when the mobile robot 1 approaches the second and third obstaclesO02 and O03, the obstacle detector 100 may input a detection signal, andthe obstacle recognition unit 111 may determine the obstacle to bepositioned within a predetermined distance and may determine theobstacle based on pre-captured image data. The obstacle recognition unit111 may determine the obstacle before the detection signal is input, maytransmit the image data to the server or the terminal, and may determinethe obstacle.

When a traveling direction of the mobile robot 1 is changed, if theobstacle is determined according to the detection signal, thepossibility that image data captured before the traveling direction ischanged does not include an obstacle at a side toward the travelingdirection is high, and thus, the obstacle may be determined based on animage after the traveling direction is changed. Thus, when the obstacleis positioned within a predetermined distance according to the detectionsignal of the obstacle detector 100, the mobile robot 1 may determinethe obstacle based on image data pre-captured in the same travelingdirection.

The mobile robot 1 may recognize an obstacle from the captured imagedata and may determine a type of the obstacle. The image acquirer 170may store image data captured while traveling, and the controller mayanalyze image with a predetermined period, and in this case, when theobstacle detector 100 determines the obstacle or the obstacle to bepositioned within a predetermined distance, the controller may determinethe obstacle using pre-captured image data.

The controller may determine information on the type, shape, and size ofthe obstacle through image data with respect to the second and thirdobstacles O02 and O03 as described above, and thus, may control thetraveling unit to perform a response motion in response thereto.

For example, when receiving a detection signal of the second obstaclewith respect to the second obstacle O02 that is a table, the controller110 may select two response motions such as avoidance and entrance afterapproach, may analyze image data, and may determine and operate any oneof response motions according to whether the mobile robot enters a spacebelow the table.

Even if tables have the same type, the tables have different sizes, thespace below the table have different sizes of, and the mobile robot havedifferent sizes, and accordingly, whether the mobile robot enters thespace below the table may be changed, and thus, the controller may set aplurality of performable response motions, for example, avoidance,entrance, or pass after approach based on a detection signal of theobstacle detector 100, and may determine to select and perform any oneof the plurality of response motions depending on the type, shape, andsize of the obstacle through image analysis.

Whether the mobile robot enters a space below a table may be determineddepending on the height of the table and the width of a table leg and atraveling direction may be determined. With respect to the same table,that is, the second obstacle O02, a response motion may be changeddepending on whether a chair such as the third obstacle O03 is present.

Thus, the mobile robot may set a plurality of response motions ofprocessing a detection signal with respect to an obstacle when thedetection signal is input, and may set any one response motion based onobstacle information determined through an image, and thus, may performdifferent response motions depending on the size or shape of theobstacle or the size of the body of the mobile robot with respect to thetable.

FIG. 9 is a diagram for explaining recognition of a detection signal ofan obstacle and recognition of the obstacle in a mobile robot accordingto an embodiment of the present disclosure. FIG. 10 is a diagram forexplaining an operation of a mobile robot depending on a type of anobstacle according to an embodiment of the present disclosure.

As shown in FIG. 9A, when detecting that an obstacle is positionedwithin a predetermined distance, the obstacle detector 100 may input adetection signal thereof.

When a pattern acquirer photographs a pattern emitted from a patternemission unit, the obstacle detector 100 may display the pattern P1emitted to the obstacle as shown in the drawing.

As a pattern is positioned above a reference line ref1, the obstaclerecognition unit 111 may determine that an obstacle with a predeterminedheight is positioned at a side toward a traveling direction.

Thus, the traveling controller may set a plurality of performableresponse motions with respect to an obstacle positioned ahead of themobile robot in response to a detection signal. For example, theresponse motion such as avoidance, approaching, entrance, or pass may beset.

When a detection signal is input, the obstacle recognition unit 111 mayanalyze pre-captured image data, that is, image data captured before thedetection signal is input and may determine an obstacle positioned at aside toward a traveling direction.

When a pattern is emitted to a lower portion of an air conditioner asshown in FIG. 9B, when a pattern is emitted to a support of a fan asshown in FIG. 9C, when a pattern is emitted to a threshold as shown inFIG. 9D, and when a pattern is emitted to a small box as shown in FIG.9E, the aforementioned pattern shown in FIG. 9A may be photographed toform an image.

Despite different obstacles, as the same pattern is detected, theobstacle recognition unit 111 may determine the obstacles based on apre-captured image.

As shown in FIG. 10 , an image captured before a detection signal isinput may be analyzed, and an obstacle positioned at a side toward atraveling direction may be determined to be a fan.

As shown in FIG. 10A, when the mobile robot 1 stores an image capturedwhile traveling and the obstacle detector 100 detects an obstacle to bepositioned within a predetermined distance, the obstacle recognitionunit 111 may analyze a pre-captured image, may recognize an obstaclephotographed in image data, and may determine information on theobstacle, such as the type, size, or shape of the obstacle.

The obstacle recognition unit 111 may analyze image data captured beforethe detection signal is input, that is, image data captured at a timepoint at which the obstacle is photographed at a farther distance than apredetermined distance.

For example, when a detection signal is input at a second distance D02,the obstacle recognition unit 111 may photograph only a portion of anobstacle at the second distance as shown in FIG. 10C, and thus, an imageshown in FIG. 10B, which is captured before a detection signal of anobstacle is input, that is, at a predetermined time before a time pointof inputting the detection signal, and the obstacle may be determined.The obstacle recognition unit 1111 may captures an image in the sametraveling direction.

When a traveling direction of the body is changed, the obstaclerecognition unit 111 may determine an obstacle from an image capturedafter the traveling direction is changed.

The traveling controller may determine that the obstacle is a fanthrough an image among a plurality of response motions based on adetection signal, as shown in FIG. 9C, and may perform a response motionthereto. For example, when the obstacle is a fan, the mobile robot maybe confined below the fan or may be put above a support of the fan, andthus, a response motion (avoidance after approaching at a predetermineddistance) may be set to avoid the obstacle without approaching the fanat 10 cm or greater, and may control the traveling unit to operateaccording to the response motion.

When the obstacle is determined as a box through an image as shown inFIG. 9E, the mobile robot may approach and collide with the box, maycheck whether the mobile robot is capable of traveling, and may then beset to avoid the obstacle. When the box is light, a position thereof maybe changed by the mobile robot, and thus, the response motion may be setto enable the mobile robot to approach and collide with the obstacle.

The controller 110 may store information on a detected obstacle asobstacle data.

The obstacle data may be data of an obstacle that is frequently detecteda plurality of detection numbers of times of the obstacle.

The traveling controller 113 may additionally determine whether anobstacle is a dangerous obstacle depending on a type of the obstacle andmay control the traveling unit to perform an operation correspondingthereto. When an operation is determined depending on a type of theobstacle, the traveling controller 113 may perform the determinedoperation, and when an operation is not separately determined, thetraveling controller 113 may vary and set an approaching degree of theobstacle depending on whether an obstacle is a dangerous obstacle. Whenthe obstacle is a dangerous obstacle, the traveling controller 113 mayoutput a predetermined sound effect or warning horn depending on a typeof the obstacle and may output voice guidance. The traveling controller113 may set a damageable obstacle such as a pot or a vase, a pet, aconfinable leg, or the like, as a dangerous obstacle.

Even if the same detection signal is input and the same obstacle ispositioned, the obstacle recognition unit 111 may perform differentresponse motions depending on the size or shape of the obstacle.

For ex ample, when a detection signal for determining a chair leg isinput, if the obstacle is determined as a chair from an image, anoperation such as avoidance, approaching, or entrance may be performeddepending on the height of the chair and an interval between legs. Theobstacle recognition unit 111 may analyze an image after the detectionsignal is input, may calculate the interval between chairs and theheight of the chair, and may determine whether the mobile robot entersthe obstacle. Thus, the traveling controller may perform any one ofresponse motions such as avoidance, approaching, and entrance.

FIG. 11 is a diagram for explaining a mobile robot and a signal flowbetween devices for obstacle recognition according to an embodiment ofthe present disclosure.

As shown in FIG. 11 , the mobile robot 1 may perform cleaning whiletraveling in a traveling zone.

The mobile robot 1 may capture an image while traveling and may storethe image as image data.

The mobile robot 1 may capture an image irrespective of whether anobstacle is detected and may recognize an obstacle positioned at a sidetoward a traveling direction from the image.

When determining the obstacle through the image, the controller 110 mayanalyze the image based on the stored obstacle information and maydetermine the obstacle (S1).

In this case, according to the first embodiment, the mobile robot 1 mayanalyze the image and recognize the obstacle while continuouslycapturing the image when the obstacle detector 100 detects the obstacle.

The controller 110 may store image data, and when the obstacle detectionunit detects the obstacle, the controller 110 may analyze pre-capturedimage data, may remove a background, and may extract features when theobstacle is positioned within a predetermined distance. The obstaclerecognition unit 111 may determine the shape, size, and type of theobstacle.

With respect to the obstacle from the image, when the obstacle isdetected to be positioned within a predetermined distance, the mobilerobot 1 may travel and avoid the obstacle depending on a type of apre-recognized obstacle through the image.

According to the second embodiment, the mobile robot 1 may store imagedata, may analyze and filter image data stored in a predetermined unitto extract features, and may determine the shape, size, and type of theobstacle. That is, even if a detection signal of an obstacle is notinput from the obstacle detection unit, the mobile robot according tothe second embodiment may recognize the obstacle included in the imagebefore the detection signal is input.

When the obstacle detector 100 inputs the detection signal in a state inwhich the obstacle is completely determined, the controller may set aresponse motion to the obstacle based on determination of the obstaclethrough the detection signal and the image and may perform apredetermined operation when the obstacle is positioned within apredetermined distance.

The mobile robot 1 according to the first and second embodiments maytransmit the image data to the server 90 or the terminal 80 and mayrequest that the server 90 check the obstacle (S2).

The server 90 may analyze image data, may extract the feature of theobstacle, and may determine a type of the obstacle based on a shapethereof. The server 90 may store accumulating data of the obstacle in adatabase and may use the data to determine the obstacle.

When recognizing a new obstacle from an image, the server 90 maygenerate a recognition model of the obstacle and may update obstacleinformation. The recognition model may include information on thefeature of the obstacle and a generated environment to determine thetype of the obstacle from the image. The recognition model may be usedto set a response motion to the corresponding obstacle, and whenreceiving images of similar obstacles, the recognition model may analyzethe feature based on a pre-generated obstacle model, may determine atype of the obstacle, and may set the response motion thereto as anavoidance operation. For example, when a specific obstacle is detected,a response motion may be set with respect to whether the mobile robotimmediately avoids a corresponding obstacle without approaching theobstacle, whether the mobile robot approaches the obstacle at apredetermined distance and then avoids the obstacle, or whether apredetermined warning horn is output.

When the mobile robot 1 is not capable of accessing a server, theterminal 80 may transmit image data received from the mobile robot tothe server. The terminal 80 may determine the type of the obstacle basedon the received image data and may determine a type of the obstaclethrough user input.

The server 90 or the terminal 80 may transmit data of a type of theobstacle to the mobile robot 1 in response to a request of the mobilerobot (S3). In addition, the server 90 or the terminal 80 may transmitdata of an avoidance operation corresponding to the type of the obstacleto the mobile robot 1. The mobile robot 1 may perform the avoidanceoperation based on the received data.

The mobile robot 1 may determine the type of the obstacle based on thereceived data and may perform the avoidance operation in responsethereto. The avoidance operation may also be set from a server or aterminal, and as necessary, any one of a plurality of operations may beselected and performed. As necessary, any one of a plurality ofoperations may be selected and input through the terminal.

FIG. 12 is a flowchart for explaining detection of an obstacle and aresponse motion in a mobile robot according to an embodiment of thepresent disclosure.

As shown in FIG. 12 , the mobile robot 1 may capture a plurality ofimages through the image acquirer 170 while traveling. The controller110 may capture an image while traveling even if any obstacle is notdetected through the obstacle detection unit.

The image may be captured as a still image at a predetermined timeinterval or may be captured as a video image through continuousphotography. The image acquirer 170 may store the captured images in aplurality of image data 101 to 103 in a data unit (S11).

The image processor included in the image acquirer 170 may filter acaptured image and may store image data at a predetermined timeinterval, and when a video image is captured, the image processor mayanalyze the video image in frame units, may remove unnecessary frames,and may then store image data.

According to the first embodiment, the controller 110 may store imagedata, and when the obstacle detection unit detects an obstacle (T1), ifthe obstacle is positioned within a predetermined distance, thecontroller 110 may analyze pre-captured image data and may remove abackground (S12), and may extract features (S13). The obstaclerecognition unit 111 may determine the shape, size, and type of theobstacle (S14).

The controller 110 may set a plurality of response motions in responseto an obstacle determined through a detection signal and an image. Thecontroller 110 may select any one the plurality of determined responsemotions and may control the traveling unit to perform the selectedresponse motion (S17).

According to the second embodiment, before the obstacle detector 100inputs the detection signal, the controller 110 may analyze and filterthe image data in predetermined units to extract features (S12 and S13)and may determine shape, size, or type of the obstacle (S14). When theobstacle detector 100 inputs the detection signal in a state in whichthe obstacle is completely determined, (T2), the controller may set aresponse motion to the obstacle and may perform a predeterminedoperation based on determination of the obstacle through the detectionsignal and the image (S17).

The controller 1110 may transmit a plurality of image data to the server(S15) and may request that the server determine the obstacle. Whenreceiving data of the obstacle from the server, the controller 110 maystore obstacle information (S16) and may determine the obstacle based onthe information (S14).

The controller 110 may set a response motion and may perform theresponse motion based on the detection signal of the obstacle detector10 and the image (S17).

FIG. 13 is a flowchart for explaining recognition of an obstacle and acontrol method based thereon in a mobile robot according to the firstembodiment of the present disclosure.

As shown in FIG. 13 , the mobile robot 1 may travel in a region of thetraveling region H, in which the mobile robot 1 is capable of traveling,and may clean a predetermined region (S310).

When receiving a moving or cleaning command, the mobile robot 1 maycapture an image through the image acquirer 170. The image acquirer 170may continuously capture an image (a video image) or may capture animage at a predetermined time interval (S320). The captured acquiredimage may be stored as image data.

The image acquirer 170 may set a photography period depending on amoving speed of the mobile robot or may set the photography period basedon a distance for detecting an obstacle by the obstacle detection unit.

With respect to a plurality of image data, the image acquirer 170 maydetermine whether an obstacle is capable of being recognized, may filterthe image data, may select an image that is capable of being analyzed,and may store the image as image data.

For example, the image acquirer 170 may select a normally captured imageand may store the image as image data except for the case in which theobstacle is not capable of being normally photographed due to movementof the obstacle or movement of the body 10 during photography, e.g., thecase in which it is not possible to recognize the obstacle due to ashaking image of the obstacle as a photography target or the case inwhich it is not possible to recognize the obstacle due to an unfocusedimage.

The image acquirer 170 may capture an image while the mobile robot 1 isoperated, and the obstacle detector 100 may emit a pattern or may detectan obstacle positioned at a side toward a traveling direction usingultrasonic waves, infrared rays, or laser beams and may input thedetection signal (S300).

The obstacle recognition unit 111 may determine whether an obstacle ispositioned at a side toward a traveling direction based on the detectionsignal or may determine the size or position of the obstacle throughpattern analysis using a 3D sensor (S340).

When determining the obstacle through the shape of a pattern obtained byemitting the pattern, the controller 110 may separate and identifyobstacles with the same detection signal being input and may set aplurality of performable response motions for respective detectionsignals (S350).

A detectable distance is changed depending on a type of a sensor of theobstacle detection unit, and thus, the controller 110 may determine adistance to the obstacle based on the detection signal and may determinewhether the obstacle is positioned at a predetermined distance (S360).

The controller 110 may determine a plurality of performable responsemotions with respect to an obstacle positioned at a side toward atravelling direction depending on a shape of the detection signal, andwhen the obstacle is positioned within the predetermined distance, thecontroller 110 may analyze image data prior to a time point ofpre-photography (S370), and may determine the obstacle (S380).

The obstacle recognition unit 111 may analyze image data, may filter theimage data depending on whether it is possible to recognize theobstacle, may remove a background from the filtered image data throughimage processing, and may then extract an outline or feature point ofthe obstacle to extract the shape and shape of the obstacle.

The obstacle recognition unit 111 may determine whether image data isidentified based on the brightness and clarify of image data, and amoving speed of the body 10. The obstacle recognition unit 111 maydetermine the brightness of the image data based on a brightness valueof a plurality of pixel values of the image data and may classify thebrightness value into excessive exposure, insufficient exposure, andnormal exposure. When the moving speed of the body is equal to orgreater than a setting speed, the obstacle recognition unit 111 maydetermine a captured image to shake, may determine the clarity of theimage data, and may determine whether it is possible to identify theimage data.

The obstacle recognition unit 111 may extract the extracted outline andfeature of the obstacle to analyze a shape of the obstacle and maydetermine the type and size of the obstacle based on obstacle data.

When there is no separate obstacle data or it is not possible todetermine a type of the obstacle from obstacle data, the obstaclerecognition unit 111 may transmit the image data to the server 90 or theterminal and may request that the server 90 or the terminal check theobstacle. The obstacle may be determined through the server immediatelywhen the image is captured. The type of the obstacle may be determinedvia autonomous determination or response from the server.

The server 90 may store a plurality of image data with respect to oneobstacle, may extract the features of a specific obstacle, and may storethe features in a database. The server 90 may analyze image datareceived from the mobile robot 1 and may compare the image data withpre-stored data, and thus, may determine a type of the obstacle. Theserver 90 may transmit a type of the obstacle and obstacle data relatedthereto to the mobile robot 1.

The obstacle recognition unit 111 may set any one of a plurality ofresponse motions based on data of an obstacle determined in response tothe shape and feature of the obstacle (S390).

The traveling controller may control the traveling unit to perform anoperation based on the set response motion (S400).

In the case of a dangerous obstacle, the traveling controller 113 mayset the traveling unit to travel to avoid the obstacle withoutapproaching the obstacle at a predetermined distance or greater. Whenthere is a predetermined operation depending on a type of the obstacle,the traveling controller 113 may perform the predetermined operation.

The traveling controller 113 may set the traveling unit to avoid anobstacle after approaching the obstacle or to immediately avoid theobstacle when a predetermined distance is reached, may set anapproaching distance when the mobile robot approaches the obstacle, andmay set the traveling unit to enter and pass through an obstacle. Inaddition, the mobile robot may also output a predetermined warning horn.

Thus, even if the same detection signal is input, the mobile robot 1 mayperform different operations depending on an obstacle via image analyze,and even if detection signals are the same and obstacles have the sametype, different response motions may be performed depending on the sizeor shape of the obstacle.

According to the present disclosure, when the mobile robot approachesthe obstacle at a predetermined distance, a predetermined operation maybe performed, and thus, the mobile robot may immediately response to theobstacle, and the obstacle may be easily determined using a pre-capturedimage, and accordingly, the obstacle may be more effectively avoided.

FIG. 14 is a diagram for explaining image transmission for recognitionof an obstacle of a mobile robot according to the second embodiment ofthe present disclosure.

The mobile robot 1 according to the second embodiment may capture aplurality of images through the image acquirer 170 while traveling. Thecontroller 110 may capture an image even if the obstacle detection unitdoes not detect an obstacle.

As shown in FIG. 14 , the mobile robot 1 may capture a plurality ofimages 301 to 304 while traveling. The mobile robot 1 may store an imagecaptured at a predetermined time interval as image data, may transmitthe image data to the server 90, and may make a request for informationon the obstacle.

The mobile robot 1 may periodically capture a still image and maycapture a video through continuous photography. The image acquirer 170may store the captured image in a data unit as a plurality of image data301 to 304.

The image processor included in the image acquirer 170 may filter acaptured image and may store the image data at a predetermined timeinterval or a predetermined moving distance interval, and when a videoimage is captured, the image processor may analyze the video image inframe units, may remove unnecessary frames, and may then store imagedata.

The mobile robot 1 may transmit image data captured while traveling tothe server according to a time sequence and may selectively transmitsome of the plurality of image data.

For example, when recognizing an obstacle at a side toward a travelingdirection from an image, the mobile robot 1 may transmit correspondingimage data to the server.

The server 90 may analyze a type of an obstacle and may transmit theobstacle information to the mobile robot 1, and the mobile robot 1 mayperform an avoidance operation on the obstacle in response to thereceived obstacle information. The avoidance operation may be anoperation of approaching an obstacle, changing a traveling direction inresponse to a type of the obstacle, and then, avoiding the obstacle, anoperation of approaching the obstacle only at a predetermined distance,or an operation of avoiding the obstacle and traveling immediately afterthe obstacle is detected. The mobile robot 1 may output a predeterminedwarning horn, sound effect, and voice guidance in response to the typeof the obstacle, and may also re-travel after a predetermined timeelapses.

When the same image is repeatedly captured within a predetermined time,the mobile robot 1 may transmit a plurality of image data capturedwithin a predetermined time to the server.

For example, when the mobile robot 1 is confined by a space below achair or a table and it not capable of traveling, the mobile robot 1 mayoutput an error, may transmit, to the server, image data captured withina predetermined time or image data captured while traveling for apredetermined distance, may receive information on the obstacle, andthus, may determine a current confinement situation. The mobile robot 1may determine that the mobile robot is not capable of traveling due to aconfinement situation and may output an error based thereon.

The server 90 may analyze the received image data and may generate arecognition model of a confinement situation of the mobile robot.

The server 90 may analyze a plurality of image data captured for apredetermined distance or a predetermined time by a mobile robot as wellas a confined positioned of the mobile robot, may analyze an imagebefore a confinement situation occurs, and may generate a recognitionmodel to enable the mobile robot to avoid the corresponding obstaclethrough the image. The server 90 may generate obstacle information basedon the generated recognition model and may transmit the information tothe mobile robot.

The server 90 may analyze the reason for a confinement situation basedon information on an obstacle around a place when the confinementsituation occurs, and information on a surrounding obstacle before aconfinement situation occurs. The server 90 may determine that aconfinement situation occurs with respect to an obstacle with a similarshape and size to the corresponding obstacle, may additionally set aresponse motion with respect to the confinement situation, and mayupdate obstacle information.

The mobile robot 1 may travel to avoid the obstacle before approachingthe obstacle in which a confinement situation occurs in response to theobstacle information received from the server and may prevent theconfinement situation from occurring.

The server 90 may analyze the obstacle in which the confinementsituation occurs to calculate statistics, may determine a candidategroup of the obstacle in which the confinement situation occurs, and mayoutput warning.

For example, a plurality of image data of a table may be analyzed, animage of a table in which a confinement situation occurs and an image ofa table in which a confinement situation does not occur may bedifferentiated therebetween and may be compared with each other, andwhen a type of the obstacle is a table, the feature of the obstacle inwhich the confinement situation occurs may be extracted, and a candidategroup of the confinement situation may be determined.

For example, when the height of the table is equal to or less than 20cm, when an interval between table legs is equal to or less than 35 cm,when another obstacle is positioned below the table, or the table hasfive legs or greater, features may be extracted, and a candidate groupof the confinement situation may be set. The server may determine aresponse motion to the confinement situation with respect to acorresponding candidate group.

As necessary, although a confinement situation does not occur, when theconfinement situation occurs in another obstacle with a similar shape,the server may generate a candidate group of an obstacle in which aconfinement situation occurs and may set a response motion to avoid thecorresponding obstacle. The mobile robot may enter the correspondingobstacle, and the server may output a warning horn or voice guidancebefore entering the obstacle and may set the response motion to enablethe mobile robot to enter the obstacle after a predetermined timeelapses.

Thus, with respect to an obstacle in which a confinement situationoccurs, when detecting the corresponding obstacle, the mobile robot 1may change a traveling direction and may travel to avoid the obstacleimmediately after approaching the obstacle at a predetermined distanceor immediately when detecting the obstacle. When avoiding the obstacle,the mobile robot 1 may output voice guidance therefor. For example, themobile robot 1 may output voice guidance ‘I am traveling and avoidingobstacle of confinement situation’.

When detecting a similar obstacle to an obstacle in which a confinementsituation occurs based on received obstacle information, the mobilerobot 1 may output voice guidance ‘If I continuously travels,confinement situation may occur’ in a state of temporary pause and maythen travel. After voice guidance, when receiving a predeterminedcommand by a user, the mobile robot may continuously travel or maychange a traveling direction and may travel in response thereto.

When an object falls while traveling, the mobile robot 1 may determine adangerous situation and may transmit a plurality of image data capturedprior thereto to the server.

The server 90 may analyze a plurality of image data, may determine anobstacle as the reason for making an object fall, may extract thefeature of the corresponding obstacle, may set a response motion, andmay then generate obstacle information.

For example, in a state in which a vase is put on a table, when themobile robot collides with the table, the vase may fall. The server maygenerate a new recognition model of the table on which the vase is put,may generate a response motion not to approach the table at apredetermined distance or greater, and may transmit the obstacleinformation to the mobile robot.

Thus, when detecting that the vase is put on the table, the mobile robot1 may travel and avoid the obstacle without approaching a predetermineddistance or greater.

FIG. 15 is a diagram for explaining a method of determining aconfinement situation through an image of a mobile robot according tothe second embodiment of the present disclosure.

As shown in FIG. 15 , the mobile robot 1 according to the secondembodiment may capture images 311 to 315 at a predetermined timeinterval and may store a plurality of image data. The mobile robot 1 maystore the image data with time information.

The image acquirer 170 may continuously capture an image or mayrepeatedly capture the image with a predetermined period. When themobile robot 1 travels for a predetermined distance, the obstacle may bephotographed in an enlarged form as the mobile robot approaches theobstacle.

The mobile robot 1 may analyze a captured image, may recognize theobstacle, may transmit the result to the server 90, and may receiveobstacle information.

While traveling, the mobile robot 1 may capture and store first to fifthimages 311 to 315 at a 0^(th) time t0, an 11^(th) time t11, a 12^(th)time t12, a 13^(th) time t13, and a 14^(th) time t14 through the imageacquirer 170 for a predetermined time interval or a predetermined movingdistance interval. For example, the mobile robot 1 may capture an imageevery moving distance of 10 cm. In addition, the mobile robot 1 maycapture an image at an interval of 10 seconds while traveling.

While traveling, when a moving distance is less than a predetermineddistance for a predetermined time, the mobile robot 1 may determine aconfinement situation. For example, when the mobile robot 1 does notmove for 5 cm or greater and a predetermined time, the mobile robot 1may determine the confinement situation.

When determining the confinement situation, the mobile robot 1 maytransmit image data that is captured before a predetermined time from atime point P11 of determining the confinement situation or image datathat is captured before a predetermined moving distance to the server90.

The mobile robot 1 may transmit an image captured prior to apredetermined moving distance, that is, 12^(th) to 14^(th) images 312 to314, based on a moving distance to the server 90. The mobile robot 1 maytransmit an image during the 11^(th) time t11 to the 13^(th) time t13,that is, the 12^(th) to 14^(th) images 312 to 314, to the server 90.

The mobile robot 1 may transmit image data and error information to aterminal or the like and may output information on error due to aconfinement situation and a current position through the terminal.

The server 90 may recognize an obstacle from each image, may analyze theobstacle as the reason for the confinement situation, and may generate arecognition model to prevent the mobile robot from approaching orentering the corresponding obstacle. The server 90 may analyze thefeature of the corresponding obstacle and may apply the same recognitionmodel to similar obstacles.

As shown in the drawing, when a confinement situation occurs below thetable, the server 90 may detect a corresponding table and may generate arecognition model with a response motion that is set to prevent themobile robot from entering a space below the table. When obstacleinformation with respect to the corresponding table is pre-generated,the server 90 may add information on the confinement situation and mayre-generate a recognition model.

The mobile robot may update new obstacle information or pre-storedobstacle information and may travel and avoid the obstacle to preventthe confinement situation from occurring. A position at which theconfinement situation occurs may be set in a pre-stored map.

Thus, during next traveling, when an image such as the 12^(th) image 312is captured, the mobile robot 1 may detect the table through an imageand may travel and avoid the obstacle after approaching the obstacle ata predetermined distance without entering the table according to aresponse motion on the corresponding table. As necessary, the mobilerobot 1 may output voice guidance for the confinement situation duringavoidance traveling.

When detecting an obstacle with a similar shape to the table, the mobilerobot 1 may travel and avoid the obstacle in the same way.

FIG. 16 is a diagram for explaining a method of controlling a mobilerobot according to the second embodiment of the present disclosure.

As shown in FIG. 16 , while traveling (S450), the mobile robot 1 mayperiodically capture an image in a traveling direction and may acquireimage information (S460). The image acquirer 170 may capture an image,may store the image, and may store the image in a data unit asrecognizable image data.

While traveling, the obstacle detector 100 may detect an obstaclepositioned at a side toward a traveling direction and may apply apredetermined detection signal to an obstacle recognition unit. Thepattern emission units 120 and 130 may emit a pattern with apredetermined shape in a traveling direction, may photograph thecorresponding pattern through the pattern acquirer 140, and may inputthe image to the obstacle recognition unit 111.

The obstacle recognition unit 111 may analyze an image input from theimage acquirer 170 to extract features and may determine a type of thephotographed obstacle based on obstacle data pre-stored in the data unit180 (S470).

The controller 110 may transmit image data to the server and may make arequest for obstacle information. The controller 110 may determine atype of the photographed obstacle in the image based on data receivedfrom the server. The controller 110 may receive and set information onat least a performable response motion on the obstacle as well as thetype of the obstacle from the obstacle information.

With respect to the pre-determined obstacle, the obstacle recognitionunit 111 may detect whether the corresponding obstacle is positionedwithin a predetermined distance through the obstacle detector 100(S480).

When the obstacle recognition unit 111 recognizes that an obstacle ispositioned within a predetermined distance, the traveling controller 113may determine the pre-recognized obstacle from the image (S490), may setany one of a plurality of response motions in response to a type of thecorresponding obstacle, and may control the traveling unit 250 to avoidthe obstacle (S500).

The traveling controller 113 may perform a response motion including atleast one combination of approaching, entrance, passing, and avoidancewith respect to the obstacle in response to the pre-determined type ofthe obstacle. The controller 110 may output a predetermined warninghorn, sound effect, and voice guidance through the output unit 190.

When a response motion is set to enable the mobile robot to enter thedetected obstacle, the traveling controller 113 may control thetraveling unit 250 to enable the body to continuously travel.

For example, when a table is detected, the traveling controller 113 maychange a traveling direction after the body approaches the body and mayenable the mobile robot to enter a space below the table and tocontinuously travel.

The obstacle recognition unit 111 may continuously analyze an imagecaptured while traveling to determine a type of the obstacle, and whenthe obstacle detector 100 detects the obstacle within a predetermineddistance, the traveling controller 113 may set a response motion of thecorresponding obstacle and may control the traveling unit.

The controller 110 may transmit image data to the server 90 and may makea request for information on the obstacle (S510).

The server 90 may analyze image data received from a plurality of mobilerobots to determine a type of an obstacle, may update information on thepre-determined obstacle, and may generate a recognition model of a newobstacle. The server 90 may transmit obstacle information or updatedobstacle information based on a recognition model that is newlygenerated according to a request of the mobile robot, to the mobilerobot.

The mobile robot 1 may store data received from a server and may updatepre-stored obstacle information. The mobile robot 1 may determine theobstacle based on new data, may transmit the determined information tothe server, and may check information on the obstacle.

When a confinement situation occurs in the pre-recognized obstacle, themobile robot 1 may transmit a plurality of image data that is capturedbefore a confinement situation occurs to the server and may make arequest for new information on the confinement situation.

The server 90 may analyze a plurality of image data, may determine asurrounding obstacle before a confinement situation occur, may identifythe obstacle as the reason for the confinement situation, and may set aresponse motion.

The server 90 may newly generate a recognition model of an obstacle inwhich a confinement situation occurs, may newly set a response motiontherefor, and may transmit obstacle information to the mobile robot.

Thus, the mobile robot 1 may change pre-stored data and may perform aresponse motion on the obstacle depending on new obstacle information.When a confinement situation occurs, the mobile robot may indicateoccurrence of the confinement situation at a corresponding position, andas necessary, may set a virtual wall.

While traveling, when the obstacle recognized from an image is anobstacle in which the confinement situation occurs, the mobile robot 1may travel and avoid the corresponding obstacle according to theresponse motion that is set when the obstacle detector 100 detects thecorresponding obstacle.

FIG. 17 is a flowchart for explaining a method of determining error of amobile robot according to the second embodiment of the presentdisclosure.

As shown in FIG. 17 , while traveling (S550), the mobile robot 1 mayperiodically capture an image in a traveling direction through the imageacquirer 170 (S560). The image acquirer may capture an image and maystore the image at a predetermined time interval or a predetermineddistance interval.

The mobile robot 1 may recognize the obstacle from the captured imagedata and may determine a type of the obstacle. The image acquirer 170may store the image data captured while traveling, and the controllermay analyze image data with a predetermined period and when the obstacleis determined or the obstacle is detected to be positioned at apredetermined distance by the obstacle detector 100, the controller maydetermine the obstacle using pre-captured image data. The controller 110may transmit image data to the server, may receive obstacle information,and may determine a type of the photographed obstacle based on theobstacle information.

The controller determines information on the type, shape, and size ofthe obstacle through image data, and thus, when the obstacle detectionunit detects the obstacle to be positioned within a predetermineddistance while traveling, the controller may control the traveling unitto perform the response motion in response thereto.

For example, when a detection signal of an obstacle is input withrespect to a table, the controller 110 may select two response motionsof avoidance after approaching and entrance, may analyze image data, andmay determine and operate any one response motion depending on whetherthe mobile robot enters a space below the table.

While traveling, the traveling controller 113 may calculate a movingdistance based on the number of times of rotation or a moving speed of awheel of a traveling unit.

When an image is captured based on a moving distance, if the mobilerobot moves for a predetermined first moving distance (S570), thetraveling controller 113 may apply a control command to enable the imageacquirer 170 to capture an image in units of a first moving distance.The traveling controller 113 may control the image acquirer 170 when themobile robot moves for a predetermined time based on a moving time.

The traveling controller 113 may determine whether a moving distance fora set time, that is, a first time is less than a set distance (a secondmoving distance) while traveling (S580).

When the moving distance for the first time is less than the secondmoving distance, if the mobile robot is not capable of normallytraveling, the traveling controller 113 may determine error. Thetraveling controller 113 may determine that the mobile robot is notcapable of moving due to a surrounding obstacle and may determine aconfinement situation (S590).

In this case, after the body enters the obstacle, the travelingcontroller 113 may determine whether the body is confined based on amoving distance per hour. For example, when a table is too low or aninterval between table legs is narrow, or when another type of obstacleis present below the table, a confinement situation in which the body isnot capable of normally traveling may occur.

When a moving distance for a set time is less than a set distance, thetraveling controller 113 may determine that the body is confined. Thetraveling controller 113 may travel and avoid an obstacle in response toa type of the obstacle.

When determining a confinement situation, the traveling controller 113may call a plurality of image data from a data unit for a previouspredetermined time or a predetermined distance and may transmit theimage data to the server 90 through a communicator (S600). A movingdistance for the first time is less than the second moving distance, andthus, the mobile robot 1 may transmit information including image databefore a confinement situation occurs, to the server.

The server 90 may analyze a plurality of received image, may analyze theimage data received from another mobile robot to classify a type of theobstacle, may calculate required information depending on a type of theobstacle, and may set each response motion.

As the server 90 performs image analysis and learning through aplurality of image data, even if tables are present, recognition modelsmay be respectively generated to perform different response motionsdepending on the shape and size of the tables, and thus, obstacleinformation may be newly generated based thereon, or pre-stored obstacleinformation may be updated. The server may set a plurality of responsemotions to be performed by the mobile robot and may provide obstacleinformation, and thus, the mobile robot may perform any one responsemotion.

The controller 110 may store obstacle information received from theserver through the communicator in the data unit (S610).

The mobile robot may travel to avoid or escape from a confinementsituation based on received data (S620). The controller may output errorwhen the mobile robot is not capable of escaping from the obstacle andmay output a warning horn. The controller may transmit a currentposition and a pre-captured image to a terminal and may enable the userto release the confinement situation of the body.

When photographing a table, the obstacle recognition unit may detect thetable in which the confinement situation occurs through an image basedon pre-stored obstacle information.

When determining that the obstacle is positioned within a predetermineddistance, the traveling controller may travel to avid the table as theobstacle based on the changed obstacle information.

That is, in the case of initial detection, when a response motion is setto enable the body to enter the obstacle and then the confinementsituation occurs, a response motion may be set as any one of avoidanceafter approaching or avoidance with respect to a corresponding obstacleand may enable the body to avoid or escape from the obstacle in the caseof next detection.

Even if tables have the same type, the tables may have different sizes,spaces below the tables may have different sizes, and the size of themobile robot may be changed, and thus, whether the mobile robot iscapable of entering the tables may be changed, and accordingly, thecontroller may set a plurality of response motions, for example, aperformable response motion to avoid, enter, or pass through theobstacle after approaching based on a detection signal of the obstacledetector 100, and in this case, may determine to set and perform any oneof the plurality of response motions depending on the type, shape, andsize of the obstacle via image analysis.

When other tables that are not the same but are similar to each otherare detected, the controller 110 may determine whether a confinementsituation occurs due to the tables and may set the mobile robot to avoidthe obstacle without entering the obstacle. The mobile robot may alsooutput a predetermined warning horn.

Thus, even if the same detection signal is input, the mobile robot 1 mayperform different operations depending on an obstacle via imageanalysis, and even if the same detection signal and the same type ofobstacles are present, the mobile robot 1 may perform different responsemotions depending on the size or shape of the obstacles.

According to the present disclosure, the mobile robot may immediatelyrespond to an obstacle through a pre-captured image, and when aconfinement situation occurs, the mobile robot may classify an obstacleas the reason for the confinement situation through a pre-captured imagewhile traveling for a predetermined time or a predetermined distance andmay travel and avoid the obstacle when detecting the correspondingobstacle, thereby preventing the confinement situation from occurring.

In addition, obstacle information may be updated by updating a newrecognition model for an obstacle through a server, and thus, the mobilerobot may avoid the obstacle in a similar situation, a confinementsituation may be prevented from occurring and the mobile robot maycontinuously travel.

While this invention has been particularly shown and described withreference to exemplary embodiments thereof, it will be understood bythose skilled in the art that various changes in form and details may bemade therein without departing from the spirit and scope of theinvention as defined by the appended claims.

[Description of reference numerals]  1: mobile robot  10: body 100:obstacle detection unit 110: controller 111: obstacle recognition unit113: traveling controller 120, 130: pattern emission unit 140: patternacquirer 150: sensor unit 170: image acquirer 180: data unit 250:traveling unit 260: cleaner

What is claimed is:
 1. A mobile robot comprising: a moveable body; animage acquirer configured to capture an image in a traveling direction;an obstacle detection unit configured to detect an obstacle positionedat a side toward the traveling direction; and a controller configured tostore a plurality of image data captured through the image acquirer, tostart a response motion at a predetermined time point at which theobstacle detection unit determines that the obstacle is positionedwithin a predetermined distance, and to determine the response motiondepending on the determined obstacle based on the image data acquiredprior to the predetermined time point, wherein the controller sets aplurality of response motions to be performed according to apredetermined type of a detection signal inputted by the obstacledetection unit, and selects any one of the plurality of response motionsbased on the image data.
 2. The mobile robot of claim 1, wherein thecontroller performs control to perform different response motions basedon the image data with respect to the detection signal that is input tobe the same.
 3. The mobile robot of claim 1, wherein the controlleranalyzes image data captured before a predetermined time based on a timepoint at which the obstacle detection unit determines that the obstacleis positioned at a predetermined distance and determines a shape andsize of the obstacle or a type of the obstacle.
 4. The mobile robot ofclaim 1, wherein the controller analyzes the image data and determinesthe obstacle before the detection signal is inputted by the obstacledetection unit.
 5. The mobile robot of claim 1, wherein, when thedetection signal is inputted by the obstacle detection unit, thecontroller analyzes the image data and determines the obstacle.
 6. Hemobile robot of claim 1, wherein the controller performs control toperform the response motion including at least one combination ofstoppage, deceleration, acceleration, reverse, stand-by, avoidance,prevention of approach at a short distance, or voice guidance.
 7. Themobile robot of claim 1, wherein the obstacle detection unit includes atleast one of an ultrasonic sensor, a laser sensor, an infrared sensor,or a 3D sensor.
 8. The mobile robot of claim 1, wherein the controllertransmits the image data to a server or a terminal, requests that theserver or the terminal check the obstacle, and determines a type of theobstacle in response to response data received from the server or theterminal.
 9. The mobile robot of claim 1, wherein the controllerdetermines whether the body is confined due to the obstacle in responseto a traveling state, and the mobile robot escapes from a confinementsituation in response to information on the obstacle, acquired from theat least one image data captured prior to the time point of determiningconfinement.
 10. The mobile robot of claim 9, wherein the controllerperforms any one predetermined response motion among a plurality ofresponse motions to prevent the body from being confined in response tothe information on the obstacle and avoids the obstacle.
 11. The mobilerobot of claim 9, wherein the controller performs any one responsemotion to output warning for confinement among a plurality of responsemotions when the obstacle is included in a candidate for causing aconfinement situation.
 12. The mobile robot of claim 9, wherein thecontroller calculates a moving distance of the body, determines that thebody is confined when a moving distance for a set time is less than aset distance, transmits at least one image data captured for a previouspredetermined time before a time point of determining the confinement orcaptured while traveling for a previous predetermine distance to aserver, and acquires the information on the obstacle.
 13. A method ofcontrolling a mobile robot, the method comprising: while traveling,capturing an image in a traveling direction and storing image data by animage acquirer; determining that an obstacle is positioned within apredetermined distance through an obstacle detection unit; determining aresponse motion depending on the determined obstacle based on the imagedata acquired prior to a predetermined time point of determining thatthe obstacle is positioned within a predetermined distance; starting aresponse motion on the obstacle at the predetermined time point; andoperating based on the response motion and traveling to avoid and passthrough the obstacle, wherein setting the response motion includessetting a plurality of response motions depending on a shape associatedwith a detection signal generated inputted by the obstacle detectionunit and selecting any one of the plurality of response motion based onthe image data.
 14. The method of claim 13, further comprising:analyzing the image data and determining the obstacle before thedetection signal is inputted by the obstacle detection unit and after animage is captured.
 15. The method of claim 13, further comprising: whenthe detection signal is inputted by the obstacle detection unit,analyzing the image data and determining the obstacle.
 16. The method ofclaim 13, further comprising: while traveling, determining whether abody is confined due to the obstacle in response to a traveling state;and when determining that the body is confined, escaping from aconfinement situation in response to information on the obstacle,acquired from the at least one image data captured prior to a time pointof determining confinement.
 17. The method of claim 16, furthercomprising: performing any one of the plurality of response motions andavoiding the obstacle to prevent the body from being confined inresponse to the information on the obstacle to prevent the body frombeing confined.
 18. The method of claim 16, further comprising:transmitting the image data to a terminal or a server; and analyzing theat least one image data, recognizing a surrounding obstacle from imagedata captured before the body is confined, determining an obstacle as areason for confinement, and generating the information on the obstacle.